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I Ins note is partK expository. Inequalities relating inversion with, respectively, extraction of principal 
submatrices and tlic Hadarnanl product in the two possible orders arc developed in a simple and unified wa\ for 
[M>sitive definite matrices. These inequalities are known, hut we also characterize the cases of equality and strict 
inequality. A by-producl is. for example, a pleasant proof of an inequality due to Fiedler. In addition, it is shown 
that the Hadamard product preserves inequalities in a generalization of Solan's observation. In the process, main 
tools for dealing with the positive semi-definite partial ordering are exhibited. 
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The inequalities we discuss are with respect to the positive semi-definite partial ordering on hermitian //- 
by-n matrices. By 

P > Q 

we mean that P — Q is positive semi-definite, and by 

P > Q 

we moan that P — (J is positive definite, hi particular, P > and P > mean, respectively, that P is 
positive semi-definite and P is positive definite. See [4] for an exposition of standard facts about positive 
definite matrices used herein. We first note that congruence preserves this ordering. 

(1) OBSERVATION: HP and Q are rc-by-n hermitian matrices, then 

P > Q implies T*PT > T*QT 

for all n-by-m complex matrices 7\ and 

P > Q implies T*PT > T*QT 

for all n-by-m complex matrices T of rank m. 

PROOF: If P - Q is positive semi-definite then y*(P - Q)y > for all y £ C n . Thus, x*(T*PT ~ 
T*QT)x = {Tx)*(P - Q)Tx > for all x G C m which, in turn, means that T*PT - T*QT is positive semi- 
definite and therefore, thai T*PT ^ T*OT. The second stafement is verified analogously. 

(2) Observation: For/i-by-n positive definite hermitian matrices/* and (J, 

P > Q if and only if ()~ l >P l 
and 

P >Q if and only if Q~ l > P~\ 



Figures in brack. -is indicate the literature references at the end of this paper. 
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PROOF: Since P is positive definite, it may be written P = T*T where T is n-by-n and nonsingular. From 
observation 1 and P > Q it follows that / > T~ 1 *(JT~ l . Since this means that / — T~ l *QT~ l is positive semi- 
definite, it follows that all eigenvalues of T~ l *QT~ l are less than or equal to 1 and, therefore, that all those 
of T(J~ l T* are greater than or equal to 1. This is to say that TQ~ X T* — I is positive semi-definite, or that 
TQ~ X T* > I. Again using observation 1, this translates into Q~ l > T~ l T~ l * = P~ 1 . This proves the first 
statement because the two implications are equivalent. An analogous argument verifies the second statement. 

A key observation is the partitioned form for the inverse of a general nonsingular matrix. 

(3) Observation: Suppose that A is an n-by-n nonsingular matrix partitioned as 




where A i is n A -by-ni and A 4 is n2-by-n 2 , n — n x + n 2 . Then 

[(AsAx-H, - Aj-UiAt- 1 (A 4 - A 3 A,-'A 2 )-> J 

partitioned conformally. This assumes all the relevant inverses exist. 

PROOF: This may be verified by direct matrix multiplication or by solving for the blocks of A ~ 1 . 

In case the matrix is positive definite hermitian, the relevant inverses do exist and the form of observation 
3 translates into 

(4) Observation: For a partitioned n-by-n hermitian positive definite matrix, 

A B\-i I (A -BC' l B*)- 1 A- l B{B*A- l B - C)~ x \ 

£* CJ \(B*A~ l B - C)- l B*A- x {C - B*A~ 1 B)~ 1 J 

where the partitioning is conformal. 

(5) Corollary: For an n-by-n by hermitian positive definite matrix partitioned as in observation 4, the 
following inequalities hold 

A>BC~ l B* and C > B*A~ l B. 

PROOF: This follows from observations 2 and 4 and the facts that the inverse of and any principal submatrix 
of a positive definite matrix must be positive definite. 

Observation (3) has been made by many authors in this and several equivalent forms. See, for example, [1] 
where many related facts are also developed. Corollary (5) has been noted also in [3] by different means and 
probably has a more extensive history. We next depart from known observations and note a fact which allows 
us to link several types of inequalities and explore more deeply and simply some observations made in [9]. 

The main result is a comparison between the inverse of a principal submatrix of a positive definite matrix 
P and the corresponding principal submatrix of P~ l (both of which are necessarily positive definite). We 
denote the set {1, 2, • • •, n} by N and then /, / C N are index sets. For an arbitrary n-by-n matrix A, we 
denote the submatrix of ,4 built from the rows indicated by I and the columns indicated by J by 

A (I, J), 

and we abbreviate A (/, /) to A (/). The complement of/ (with respect to N ) is written / 

(6) Theorem: For an n-by-n positive definite matrix P we have 

P-\I) >P(I)-\ 

Furthermore, rank (P _1 (/) — P (/) _1 ) = rank P (/, /' ), so that the above inequality is strict if and only if P (/, 
/' ) has full row rank and equality holds precisely when P (/, /' ) =0. 

586 



Remark: Statement (6) may be paraphrased, "The inverse of a principal submatrix is less than or equal to 
the corresponding principal submatrix of the inverse." 

Proof of (6): Because the statement in question is invariant under permutation similarity, it follows from 
observation (4) that 



Therefore, 



and 



P-\I) = (P(I) -P(l* V)P(I')~ l P(I % /')*) 



(p-\i))- 1 = />(/) - P(/, V)P(i'Y x P(i % /')*, 



p(D - (p- l (i)y i = />(/, r)P(i')- i P(i, /')*. 



Since P(l') is positive definite by virtue of being a principal submatrix of a positive definite matrix, the 
right-hand side of the equality just above is positive semi-definite (and, in fact, has rank equal to that ofP(/, 
/')). We then have that P(I) > (P'H/))" 1 and, employing observation 2, P '(/) > P(I)'K The cases of 
equality and strict inequality are, clearly, determined by P(/, /' ) as asserted, and an analogous argument 
passing from P' 1 toP shows that rank (P" 1 ^) ~ P(D~ l ) = rankP(/, /'). 

REMARK: We note that the inequality of theorem 6 cannot be strict whenever the cardinality of/ is greater 
than i n . 

The Hadamard product oftwom-by-n matrices A = (o#) and B — (6y) is the m-hy-n matrix 

AoB = C = (c„) 

where r,j = a,j6,j for / = 1, • • • , m; j = 1, • ■ • , n. Further, the Kronecker product of an mi-by-nj matrix /4 
= (a<j) and an m 2 -by-n 2 matrix 5 is the m x m 2 -hy-n x n 2 matrix 




(7) OBSERVATION: The Hadamard product of two matrices (for which it is defined) is a submatrix of the 
Kronecker product of those two matrices, and if the two matrices are n-by-n, the submatrix is principal. 

Proof: This is immediate, and in the n-hy-n case we actually have 

AoB = {A® B)(I) 

where/ = { 1, n + 2, 2n + 3, • • •, n 2 }. 

(8) Observation: If H > (>)£are n-by-n hermitian matrices, then 

for any index set /C/V. 

PROOF: This simply follows from the fact that // — K positive semi-definite (positive definite) implies ( // 
— K)(I) = //(/) ~ K(I) is also [4]. Observations 7 and 8 are well known and provide a useful device for 
deducing inequalities for Hadamard products from those for Kronecker products (which, although stronger, 
are often easier to see) since H ® K is positive definite when H and K are |8|. We note also the well-known 
result attributed to Sehur that the Hadamard product of positive definite matrices is positive definite [6] 
(which is a classic illustration of the comment just made). See also [8] for manipulative facts relating to these 
products. 
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The next main result deals with Hadamard products and inverses of positive definite matrices. 
(9) Theorem. For n-by-n positive definite hermitian matrices H and K: 

tf-iotf-i > (HoK)-\ 
Moreover, rank (H~ 1 °K~ 1 — (H°K)~ l ) = rank L where 





Thus, the inequality is strict if and only if rank L = n, and equality holds if and only if hi °KP t = 0, t = 1, 
• • •, n - 1. 

PROOF: Note that (H (g) £) _1 = Z/" 1 ® A. -1 and the proof is a direct application of theorem 6 to // (g) £ 
using observations 7 and 8 and the fact that (H (g) £)(/, V) — L where / is as in the proof of observation 7. 

(10) COROLLARY. Equality holds in the inequality of Theorem 9 if and only ifW and K are both diagonal, 

PROOF: If both are diagonal, equality is clear, and, on the other hand, H°KP f = for t = 1, • • • , n — 1 
only when// and K are diagonal (for positive definite matrices), so that the converse follows from the condition 
for equality in theorem 9. 

The inequality of theorem 9 is known (see, for example [9] for variations); but the cases of equality and 
strict inequality seem not to have been treated, and the deduction from theorem 6 is especially simple. This, 
of course, shows the link between the Hadamard type inequality (9) and the partitioned inequality (6) and in 
some sense it explains why the otherwise rather mystifying inequality (9) is true. We note that the cases of 
equality and strict inequality have nothing to do with unitary invariants but are rather more combinatorial. 

For a matrix A, denote thep-th Hadamard "power" 



by A (p \ Several special cases of theorem (9) are worth noting. 

(11) COROLLARY: For a positive definite n-by-n hermitian matrix H, 

//-lo//-l > (//o//)-l. 

Inductively, (11) may be generalized using (9) and using the fact that A > B > and C > D > Q hermitian 
imply that A°C ^ B°D. (The latter is a special case of a statement to be proved later.) 

(12) COROLLARY: For a positive definite n-by-n hermitian matrix H. 

(ff-l)<P>> (#<»>)-! 

for all p = 1, 2, 3, • • •. 

Taking K = // -1 , and noting the commutativity of the Hadamard product, produces another interesting 
special case of (9). 

(13) Corollary: For a positive definite n-by-n hermitian matrix H, 

HoH~ l > (#o#-i)-i. 

Statement (13) is of particular interest in that it yields an immediate proof of an inequality due to Fiedler [2]. 
Note that (13) states that a certain matrix is greater than or equal to its inverse; therefore, that matrix must be 
greater than or equal to the identify. 
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(14) Corollary: For a positive definite n-hv-n hermitian matrix H, 

(//o//-i) >/> (//o// i)-i. 

(We note that this means that all eigenvalues of H°H" 1 arc at least one.) Actually, either half of inequality 
(14) is equivalent to the other and, in turn, equivalent to (13). Fiedler's proof of H°H ' ^ / is quite 
computational, so that the approach herein provides an attractive alternative. This inequality may he 
interpreted as saving that the Hadamard product (of// and// -1 ) dominates the usual product. Il would he 
interesting to know if there are other instances of such a phenomenon. For another view of this inequality see 

[7]- 

The cases of equality in (1 1)-(14) are, of course, entirely covered by (10). Strict inequality can never attain 
in (13) or ( 14) because the row sums of H°H~ l are all equal to one, and thus 

e r (//o//-i) 6 = e Tj e = ( > T {HoH~ x r\> = n. 

Strict inequality can occur in ( 1 1 ) and (12) and, in fact, is likely. (We conjecture thai strict inequality holds 
in ( 1 1 ) and ( 12) unless // has a row with ( // - 1 ) entries equal to zero. ) 

Inequality (II) is of an intriguing type: two matrix operations (in this case inversion and Hadamard 
squaring) commute except for an inequality. There arc other examples of this phenomenon (e.g., see [5] ), and 
a more general understanding of il would he worthwhile. Inequality (6) is, of course, also of this type. 

We turn finally to a different sort of Hadamard product inequality which generalizes Schur's theorem (the 
Hadamard product of two positive definite matrices is positive definite) and points out that matrices under 
Hadamard multiplication are quite analogous to complex numbers in rectangular coordinates as far as simple 
multiplicative inequalities. For an arbitrary n-hx-n complex matrix define 

H(A) m 1 / 2 (A +A*) 

the "hermitian part" (or real part) o£A. A natural extension to all matrices of the partial ordering for hermitian 
matrices discussed so far is the following. Forrz-by-A* matrices A, B we write 

A >B 

if and only if H{A ) 5: H{B) in the positive semi-definite partial ordering on hermitian matrices. (Note that A 
> B and B > A do not imply A = B, only that H(A) — H {B).) We make the analogous extension for strict 
inequality. A straightforward calculation yields 

(15) Observation: For two ra-by-n complex matrices A and B, A > B if and only if Re(x*Ax) ^ 
Re(x*Bx) for all xG C n . 

Of the observations made earlier for the positive semi-definite partial ordering on hermitian matrices, those 
which carry over to the extended partial ordering (under obvious interpretation) are (1), (2), and (5). It is 
obvious that A > B and C > D imply A + C ^ B + D. 

(16) EXAMPLE: That theorem 6 does not in general extend to all matrices satisfying H(A ) X) is shown by 




Then, 




and V2 £ 1. 

Our principal observation is: 
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(17) Theorem: Let P > Q be n-by-n positive semi- definite hermitian matrices, and suppose that A > B > 
ar^ n-6y-n complex matrices. Then 

PoA >QoB. 

PROOF: We assume, without loss of generality, that P is positive definite because then the positive semi- 
definite case follows by a continuity argument. Because of observations 7 and 8, the desired inequality holds 
if 

P® A>Q®B 

and, because of the extension of observation 1, this inequality holds if there is a nonsingular n 2 -hy-n 2 matrix 
R such that 

R*(P® A)R >R*(Q® B)R. 

By virtue of the assumption that P is positive definite, there is a non-singular n-by-n matrix T such that 

T*PT = D and T*QT = E 

are simultaneously diagonal, and, again because of observation 1, D 32: E ^ (i.e., da ^ en ^ for all i = 
1, ••-,»). We now take/? = T ® /, and, then 

/?*(/> ® .4)/? = (7 1 * <g) /)(P<g> i4)(r® /) = T*PT® A = D® A 

and, similarly 

R*(Q® B)R = E® B. 

Thus it suffices to show that 

D® A>E® B. 



However, if we partition an arbitrary x G C n into .r = [ ] where each x x Ei C n * i = 1, • * • , rc, then 




x*(D ® A)x = 2, d u xfAxi and %*(£ ® 5)% = 2 euxfBxi. 

Since Re(x^ Ax { ) > Re{xfBx { ) ^ and J„ > e tt > 0, i = 1, • • • , n, we have 

Re(x*(D®A)x) >Re(x*(E® B)x) 
and D ® A > E ® B by observation 15. Thus 

and the proof is complete. 

It is clear that theorem 17 may be extended to the case of strict inequality which we mention without proof. 

(18) OBSERVATION: For n-by-n hermitian matrices P > Q > and n-hy-n complex matrices A > B > 
0, PoA > Q°B. 

Finally, (17) mav be extended to the case in which none of the factors is hermitian under proper 
assumptions. This completes the analogy to multiplication of complex numbers. Define 
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S(A) m V 2 (4 -.4*) 

the "skew-hermitian part' 1 of an n-hy-n complex matrix .1. Then A = //( 1 ) + S(A ) and iS(A ) is hermitian. 
Now for four matrices, ,4, H, C, D a calculation yields thai 

H(AoC) = H(A)o//(C) + S(A)oS(C) 

and 

H{BoD) = H(B)oH(D) + S(B)oS(D). 

Therefore, ,4°C > #°/J if and only if 

H(A)°H(C) + S(A)oS(C) >H(B)oH(D) + S(B)°S(D). 

Theorem 17 treats the case in which S(A) = S(B) = and may he used to exhibit several more 
circumstances in which A°C s£ B°D. We list some of these as examples and each may be proved quite 
formally with the aid of ( 17). 

(1 ( J) OBSERVATION: For n-by-ti complex matrices 4, #, C, and I), each of the following sets of conditions 
is sufficient for 

AoC>BoD: 

(a) H(A)>H(B)>0; H(C) >//(/;)> 0; iS(B) > iS(A) > 0; and &(£>) > iS(C) > 0. 

(b) //(4) >//(£) >0; H(C)>H(D)>0; > iS(B)> iS(A); and iS(C) > i5(D) > 0. 

and 

(c) //(4 )>-H(B) >0; and ff(C) >-#(/))> 0; 

-iS(/i ) > &(£) > 0; and lS(C) > -&(/)) > 0. 
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